Projected Sequential Gaussian Processes: A C++ tool for interpolation of heterogeneous data sets

Within MUCM there might occasionally arise the need to use large training set sizes, or employ observations with non-Gaussian noise characteristics or non-linear sensor models in a calibration stage. This technical report deals with Gaussian process models in these non-Gaussian, and / or large data set size cases. Treating such data within Gaussian processes is most naturally accomplished using a Bayesian approach, however such methods generally scale rather badly with the size of data set, and require computationally expensive Monte Carlo based inference in non-Gaussian settings. Recently within the machine learning and spatial statistics communities many papers have explored the potential of reduced rank representations of the covariance matrix, often referred to as projected or fixed rank approaches. In such methods the covariance function of the posterior process is represented by a reduced rank approximation which is chosen such that there is minimal information loss. In this paper a sequential Bayesian framework for inference in such projected processes is presented. The observations are considered one at a time which avoids the need for high dimensional integrals typically required in a Bayesian approach. A C++ library, psgp, which is part of the INTAMAP web service, is introduced which implements projected, sequential estimation and adds several novel features. In particular the library includes the ability to use a generic observation operator, or sensor model, to permit data fusion. It is also possible to cope with a range of observation error characteristics, including non-Gaussian observation errors. Inference for the covariance parameters is explored, including the impact of the projected process approximation on likelihood profiles. We illustrate the projected sequential method in application to synthetic and real data sets. Limitations and extensions are discussed.